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Abstract 

We examined effects of research-based early writing intervention delivered within a data-based 
instruction (DBI) framework for children with intensive needs. We randomly assigned 46 students 
with and without disabilities in Grades I to 3 within classrooms to either treatment or control. 
Treatment students received research-based early writing intervention within a DBI framework 
for 30 min, 3 times per week, for 12 weeks. Control students received business-as-usual writing 
instruction. We measured writing performance using curriculum-based measures (CBM) and 
Woodcock Johnson III Tests of Achievement (WJ III). We found significant treatment effects 
on CBM outcomes (Hedges g= 0.74 to 1.36). We also found a significant interaction between 
special education status and condition on the WJ III favoring treatment students with disabilities 
(Hedges g- 0.45 to 0.70). Findings provide preliminary support for using a combination of 
research-based intervention and DBI with students with intensive writing needs. 


Children at risk or identified with academic 
disabilities require high-quality, effective 
intervention to experience success in school. 
Thus, researchers have made significant 
investments in the development and evalua¬ 
tion of interventions to improve students’ 
academic performance (Chard et al., 2008; 
Zumeta, 2015). Although such interventions 
have benefited many children, a small 
proportion—approximately 5% of the student 
population—does not show sufficient res¬ 
ponse to generally effective research-based 
approaches (National Center on Intensive 
Intervention, 2013; Wanzek & Vaughn, 
2009). These children likely need more 
intensive, individualized instruction (D. Fuchs, 
Fuchs, & Vaughn, 2014; Zumeta, 2015). 

Intensive, individualized instruction was 
intended to be a cornerstone of special 
education, yet it is not widely implemented 
in current practice (D. Fuchs, Fuchs, & 
Stecker, 2010). Thus, D. Fuchs et al. (2010) 


urged the field to bring the “unique and 
effective instructional approach . . . known 
as data-based instruction [DBI] . . . back to 
the future” (p. 318) of special education. In 
this article, we describe one attempt to do so 
by using DBI in combination with research- 
based intervention to improve outcomes for 
children who experience difficulties learning 
to write. 

DBI 

DBI is a hypothesis-driven, empirical approach 
to individualizing instruction (Deno & Mirkin, 
1977) that entails (a) a systematic process (not 
a single intervention); (b) an ongoing cycle of 
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implementation incorporating assessment and 
intervention; (c) intervention delivered in 
addition to, or instead of, core instruction and 
small-group intervention; and (d) implementa¬ 
tion for a long period of time (Danielson & 
Rosenquist, 2014). A series of DB1 steps 
(described in Method section) guides teachers 
to use data to individualize instruction by 
determining when and whether instructional 
changes are needed (Danielson & Rosenquist, 
2014; D. Fuchs et al„ 2010). 

Research shows positive evidence of DBI 
on teachers’ instructional planning and 
students’ performance (Stecker, Fuchs, & 
Fuchs, 2005). Teachers who used DBI made 
more frequent instructional changes and 
identified appropriate targeted skills for 
students than teachers who did not use DBI 
(Capizzi & Fuchs, 2005; D. Fuchs, Deno, & 
Mirkin, 1984; L. Fuchs, Fuchs, & Hamlett, 
1989; Stecker & Fuchs, 2000). Further, DBI 
has resulted in significant academic improve¬ 
ments of students at risk or with disabilities 
in reading, mathematics, and spelling (see 
Stecker et al., 2005, for a review). To date, 
no research has been conducted on DBI in 
writing. 

Need for DBI in Early Writing 

Writing skills are essential for school and 
vocational success (Graham & Perin, 2007). 
Students with or at risk for disabilities are 
especially likely to experience difficulty with 
writing (Graham, Harris, & Larsen, 2001). 
For example, Salahu-Din, Persky, and Miller 
(2008) found that, nationally, only 6% of 
eighth graders with learning disabilities 
reached proficiency in writing; 46% were 
below the basic level. For students who 
struggle with writing, early writing interven¬ 
tion is essential to prevent long-term academic 
failure (Berninger, Nielson, Abbott, Wijsman, 
& Raskind, 2008). To provide effective early 
intervention for students with intensive needs, 
extensions of DBI research that include 
a focus on early writing skills are needed. 
Fortunately, crucial components needed for 
such research do exist, including early writing 
intervention and assessment. 


Early Writing Intervention 

DBI should begin with instruction that targets 
the specific needs of students with academic 
difficulties and has evidence of efficacy for 
improving academic outcomes (Stecker et al., 
2005). To target specific early writing needs, 
researchers have relied on a theoretical model 
of writing called the simple view of writing 
(Berninger & Amtmann, 2003) that specifies 
three components of writing: transcription, 
text generation, and executive functioning 
(Berninger & Winn, 2006). Each component 
is constrained by attention and memory. As 
lower-level writing skills (transcription) 
become automatic, more cognitive resources 
are available for higher-level writing skills 
and cognitive processes (text generation and 
executive functioning), which should lead to 
improved overall writing proficiency (e.g., 
Alves & Limpo, 2015). 

To date, no research has been 

conducted on DBI in writing. 

Findings from early writing intervention 
research support this theoretical model. Spe¬ 
cifically, researchers have demonstrated that 
explicit early writing interventions targeting 
basic transcription skills (e.g., handwriting 
and spelling) can lead to improved writing 
fluency and quality for students identified as 
at risk or with disabilities (for reviews, see 
Datchuk & Kubina, 2012; Graham, McKeown, 
Kiuhara, & Harris, 2012; McMaster, Kunkel, 
Shin, Jung, & Lembke, in press). Thus, both 
theory and empirical evidence indicate that 
intervention focusing on development of tran¬ 
scription skills is crucial to prevent future 
writing difficulties. 

Early Writing Assessment 

A fundamental assumption of DBI is that 
although research-based interventions may 
benefit many students, a teacher can only 
hypothesize that a given approach will be 
effective for an individual student and must 
collect ongoing assessment data to determine 
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intervention effects for that student (Deno & 
Mirkin, 1977). Curriculum-based measure¬ 
ment (CBM; Deno, 1985) has unique features 
that meet this need. First, CBM is a “general 
outcome measurement” approach (L. Fuchs & 
Deno, 1991), in that it provides an index of 
overall academic proficiency. Second, CBM has 
been demonstrated to produce reliable and valid 
scores (see reviews by Foegen, Jiban, & Deno, 
2007; McMaster & Espin, 2007; Wayman, 
Wallace, Wiley, Ticha, & Espin, 2007). Third, 
CBM is characterized by standardized admin¬ 
istration and scoring procedures, multiple 
forms with equivalent difficulties, and time 
efficiency (Stecker et al., 2005). 

CBM in writing (CBM-W) has evidence of 
producing reliable scores that serve as valid 
indicators of students’ early writing perfor¬ 
mance and are sensitive to growth over time 
(McMaster, Ritchey, & Lembke, 2011). 
Specifically, Word Dictation (a word-level 
task) and Picture Word (a sentence-level task) 
have been demonstrated to be appropriate for 
monitoring children’s early writing progress 
(Lembke, Deno, & Hall, 2003; McMaster, Du, 
et al., 2011). An important next step is 
to determine whether, when teachers use 
CBM-W for individualized instructional 
decision making, student outcomes improve. 

Purpose and Research 
Questions 

As we have described, researchers have pro¬ 
vided promising evidence supporting (a) the 
use of DBI, (b) the implementation of early 
writing interventions focusing on transcrip¬ 
tion, and (c) the use of CBM-W to obtain 
reliable, valid early writing assessment data. 
In this study, we combined these three compo¬ 
nents to examine the effects of early writing 
intervention delivered within a DBI frame¬ 
work, using CBM-W data for ongoing instruc¬ 
tional decision making. In doing so, we 
intended to shed light on whether DBI, which 
has been shown to improve student outcomes 
in reading, math, and spelling, would also 
show promise in writing. 

Thus, we examined the effects of research- 
based early writing intervention delivered 


within a DBI framework for children with 
intensive needs. Because many children in 
early elementary grades have not yet been 
identified with disabilities (Beminger et al., 
2006), we included students at risk as well 
as those already identified with disabilities. 
In addition, to provide further evidence 
regarding the effects of transcription interven¬ 
tions on overall writing skills, we examined 
the effects of intervention on both basic and 
more complex writing skills. 

Specific research questions included the 
following: 

1. What are the effects of research-based 
early writing intervention delivered 
within a DBI framework for students 
in Grades 1 to 3 at risk or identified 
with disabilities? 

2. Do effects vary by (a) special education 
status and (b) type of writing skills as 
measured by CBM-W and a stan¬ 
dardized writing measure? 

We hypothesized that students who received 
research-based early writing intervention with 
DBI would outperform control students in 
their writing achievement, on both basic and 
more complex writing skills, given theoretical 
and empirical evidence indicating that tran¬ 
scription skills can lead to improved overall 
writing proficiency (Datchuk & Kubina, 2012; 
Graham et al., 2012; McMaster et al., in press). 

Method 

Our study took place in three elementary 
schools in a large, urban midwestern district. 
Three general education teachers and four 
special education teachers from seven class¬ 
rooms agreed to participate. Teachers nomi¬ 
nated students who they believed were in need 
of intensive, individualized writing interven¬ 
tion (n= 66). Students who had parental 
consent and assented to participate (n = 50) 
were screened using CBM-W Picture Word 
and Woodcock-Johnson III Tests of Achieve¬ 
ment (WJ III; Woodcock, Mather, & McGrew, 
2001) writing subtests (Spelling, Writing 
Fluency, and Writing Samples). 
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Students were initially deemed eligible for 
the study if their CBM-W median score was 
below 15 correct word sequences (CWS), a 
cut score with evidence of classification accu¬ 
racy (Jung & McMaster, 2012). This cut score 
yielded a sample of n = 41. To maximize the 
number of students who might benefit from 
the intervention, two additional screening 
criteria were applied to students whose 
median score was greater than 15 CWS. First, 
students were included if their score on the 
WJ III Spelling subtest was below one 
standard deviation of the normative mean 
(n = 5). Because the intervention was designed 
to improve transcription skills, the Spelling 
subtest was the focus; however, eligible 
students also showed low performance on WJ 
III Writing Fluency and Writing Samples 
(below the 8th and 2nd percentile ranks, 
respectively). Second, students were included 
if they generated simple, repetitive sentences 
(e.g., “I have a dog,” “I have a lamp”) on more 
than than 50% of their CBM-W sentences 
(n= 2). These additional criteria yielded a 
final sample of n= 48. 

Of the 48 participants, 29 were students 
with disabilities receiving special education 
services, and 19 were designated “at risk” 
(low performing in writing but not identified 
with disabilities). Students receiving special 
education services represented a wide range 
of disability categories (see Table 1) and, 
according to their special education teachers, 
had intensive needs related to writing devel¬ 
opment. The largest-represented category 
was “students needing alternative program¬ 
ming” (SNAP). Students identified as SNAP 
have significant academic needs and are 
identified through a problem-solving model, 
which the participating district has used since 
the early 1990s as an alternative to the tradi¬ 
tional IQ-achievement discrepancy approach 
to identifying students with learning disabili¬ 
ties (Marston, Muyskens, Lau, & Canter, 
2003). In this approach (which has served as 
a model for some contemporary response-to- 
intervention approaches), students are identi¬ 
fied as needing special education services 
based on their responsiveness to increasingly 
intensive and individualized intervention, as 


indicated by progress-monitoring data. All 
students with disabilities in this study were 
receiving individualized academic instruc¬ 
tion in a resource room or self-contained 
special education classroom. Students identi¬ 
fied as at risk received literacy instruction in 
the general education classroom at the time of 
this study. 

All participants were assigned randomly 
within classroom to treatment or control. 
Two treatment students did not complete the 
study (described in Attrition under Results). 
Thus, complete data were available for 46 
students (27 with disabilities and 19 at risk). 
Demographic information is presented in 
Table 1. There were no statistically significant 
differences between conditions on any of the 
demographic variables. 

Measures 

We chose Picture Word for screening, pre- and 
posttesting, and monitoring progress as it has 
shown sufficient technical adequacy in terms 
of reliability, validity, and sensitivity to 
growth (McMaster, Du, et al., 2011). Each 
prompt consists of nine words with three 
words per page. Students generate as many 
sentences as possible using words presented 
with pictures for 3 min. Writing samples are 
scored using total words written (WW), words 
spelled correctly (WSC), CWS (the total 
number of adjacent, correctly spelled words 
that are grammatically correct within the 
context of the sample; Videen, Deno, & 
Marston, 1982), and correct minus incorrect 
word sequences (CIWS). Alternate-form 
reliabilities have been reported to be moderate 
to strong (r= .44 to .79), and criterion validity 
has ranged between r= .23 and r= .54 with 
the Test of Written Language (Hammill & 
Larsen, 1996; McMaster, Du, et al., 2011; 
McMaster, Du, & Petursdottir, 2009). 

We used WJ 111 (Woodcock et al., 2001) 
Spelling, Writing Fluency, and Writing Samples 
subtests for pre- and posttest. For Spelling, 
students write letters or words that the examiner 
dictates. Items are scored as 1 if correct and 0 if 
incorrect. Writing Fluency requires students to 
generate sentences quickly and accurately 
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Table 1 . Student Demographics. 

Variable 

T reatment (n 

= 22) 

Control (n 

= 24) 

x 2 

P 

n 

% 

n 

% 

Age 





4.61 

.20 

6 years 

4 

18.2 

5 

20.8 



7 years 

7 

31.8 

13 

54.2 



8 years 

7 

31.8 

2 

8.3 



9 years 

4 

18.2 

4 

16.7 



Sex 





0.55 

.46 

Male 

16 

72.7 

15 

62.5 



Female 

6 

27.3 

9 

37.5 



Ethnicity 





2.65 

.62 

American Indian 

1 

4.5 

0 

0.0 



African American 

14 

63.6 

14 

58.3 



Asian 

2 

9.1 

1 

4.2 



Hispanic 

1 

4.5 

3 

12.5 



White 

4 

18.2 

6 

25.0 



FRL 





0.46 

.50 

No FRL 

2 

9.1 

1 

4.2 



Receives FRL 

20 

90.9 

23 

95.8 



SPED 





0.00 

.96 

No IEP 

9 

40.9 

10 

41.7 



Has IEP 

13 

59.1 

14 

58.3 



Disability Categories 





5.07 

.65 

ASD 

4 

30.8 

4 

28.6 



SNAP 

6 

46.2 

5 

35.7 



S/Ll 

1 

7.6 

0 

0.0 



OHD 

1 

7.6 

3 

21.4 



PI 

0 

0.0 

1 

7.1 



DCD-MM 

0 

0.0 

1 

7.1 



EBD 

1 

7.6 

0 

0.0 



ELL status 





0.95 

.33 

Non-ELL 

15 

68.2 

13 

54.2 



ELL 

7 

31.8 

1 1 

45.8 



Home language 





5.96 

.54 

Arabic 

0 

0.0 

1 

4.2 



Amharic 

1 

4.5 

0 

0.0 



English 

14 

63.5 

13 

54.2 



Hmong 

1 

4.5 

1 

4.2 



Nepali 

1 

4.5 

0 

0.0 



Spanish 

1 

4.5 

3 

12.5 



Somali 

3 

13.6 

6 

25.0 



Tigrinya 

1 

4.5 

0 

0.0 




Note. FRL= free or reduced-price lunch; SPED= special education status; IEP= individualized education program; 
ASD= autism spectrum disorder; SNAP= student needing alternative programming; S/Ll = specific language 
impairment; OHD= other health disabilities; Pl= physical impairment; DCD-MM = developmental cognitive disabilities: 
mild to moderate; EBD= emotional and behavioral disabilities; ELL= English language learner. 
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within 7 min in response to a picture stimulus 
and three words. Each sentence is scored 1 for 
being a complete and “reasonable sentence” 
(Mather & Woodcock, 2001, p. 52) that includes 
the three words or 0 for not meeting these 
criteria. For Writing Samples, students write 
words or sentences in response to a picture or a 
verbal stimulus. Scores range from 0 to 2 based 
on scoring criteria described in the examiner’s 
manual (Mather & Woodcock, 2001). 

An aggregated score of the three writing 
subtests yields a Broad Written Language 
cluster, representing a comprehensive measure 
of written language achievement. The raw 
scores of each subtest and the aggregated score 
were converted to W scores using the WJ III 
Compuscore and Profiles Program (Schrank & 
Woodcock, 2001). Reported median reliability 
coefficients are r= .89 for Spelling, r= .86 for 
Writing Fluency, r= .84 for Writing Samples, 
and r= .94 for the Broad Written Language 
cluster (Mather & Woodcock, 2001). 

Fidelity of DBI was assessed using a modi¬ 
fied version of the Accuracy of Implementation 
Rating Scales (AIRS; D. Fuchs et al., 1984), 
focusing on administering, scoring, and graph¬ 
ing CBM-W and using CBM-W outcomes. 
Fidelity of early writing intervention was 
checked using writing intervention checklists 
developed by the first author. On both check¬ 
lists, each item was rated as 1 (observed) or 0 
(not observed), along with observation notes. 

Early Writing Intervention 

We adapted the research-based early writing 
intervention from the Center on Accelerating 
Student Learning (CASL) handwriting and 
spelling program. (Graham & Harris, 1999, 
2006). The CASL manual includes seven 
handwriting and spelling activities supported 
by research (Graham, Harris, & Chorzempa, 
2002; Graham, Harris, & Fink, 2000), 
described subsequently. 

During Phonics Warm-Up, students identi¬ 
fied sounds and locations (beginning, middle, 
end) of target letters in words. The instructor 
would say, “The first sound of [apple] is /a/. 
What sound? What letter makes the /a/ 
sound?” During Alphabet Practice, students 
practiced writing letters and words with the 


letters. The instructor modeled writing each 
target letter by tracing it with a finger. 
Students traced and copied the letters and then 
wrote the letters from memory. 

During Word Building, students generated 
words by adding a letter to the beginning of 
rimes. The instructor modeled how to make a 
real word using a rime on a card (e.g., at) 
by adding a letter in front of the rime. After 
modeling, students made as many real words 
as possible. During Word Study, students studied 
spelling words through five steps: (a) Say the 
word and its letters, (b) say the word and 
letters from memory without looking, (c) say 
the letters again, (d) write the word from mem¬ 
ory, and (e) check to see if the word is correct. 

During Alphabet Rockets, students copied 
sentences with accuracy and fluency. Students 
wrote the sentence, which included target 
letters covered in Alphabet Practice, as many 
times as possible for 3 min. During Writing, 
students wrote a story based on a story prompt 
using handwriting and spelling skills practiced 
so far. Last, during Word Sort, the instructor 
demonstrated how to categorize words by 
target features, showed how to check, and then 
had students state a rule for the patterns 
observed. Students then sorted the words. 

The writing activities were delivered over 
three sessions and then repeated with new 
content. Session 1 included Phonics Warm-up 
(5 min), Word Building (15 min), and Alphabet 
Rockets (7 min). Session 2 included Alphabet 
Practice (7 min), Word Study (10 min), and 
Writing (10 min). Session 3 included Word 
Sort (27 min). At first, tutors delivered each 
writing activity for the prescribed time (as 
indicated in parentheses after each activity) to 
all students but could then adjust time spent on 
each activity based on students’ individual 
needs after collecting at least eight CBM-W 
data points (and by following the DBI steps 
described in the following paragraph). Regard¬ 
less of how much time was spent on specific 
activities, intervention sessions were delivered 
for a total of 30 min per session. 

DBI Steps 

The first author trained three undergraduate 
and two graduate tutors to implement the 
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following eight DBI steps (Deno & Mirkin, 
1977; McMaster et al., 2014). Step 1 was 
“establish present level of performance.” 
Tutors administered and scored three CBM-W 
prompts and graphed the median baseline 
score. Step 2 was “set an ambitious long-term 
goal.” Tutors multiplied the total number of 
intervention weeks by the weekly growth 
rates expected for typically developing students 
(0.5 to 1.0 CWS per week; McMaster, Du, 
et al., 2011), and added the baseline score. 
Then, they drew a goal line from baseline to 
the long-term goal on the graph. 

Step 3 was “implement high-quality 
instruction with fidelity.” Tutors implemented 
the early writing intervention activities 
described previously. Step 4 was “monitor 
student progress toward the goal.” CBM-W 
progress monitoring data were collected twice 
weekly. Tutors added a trend line representing 
the student’s growth rate after collecting eight 
data points (as recommended by McMaster, 
Du, et al., 2011). Step 5 was “use decision 
rules to evaluate student progress and instruc¬ 
tional effectiveness.” Tutors made instructional 
decisions by comparing the trend line to the 
goal line. If the trend line was steeper than the 
goal line, they increased the goal. If the trend 
line matched the goal line, they kept the 
intervention as is. If the trend line was less 
steep than the goal line, they changed instruc¬ 
tion by implementing Steps 6 and 7. 

For Step 6, “generate hypotheses to individ¬ 
ualize instruction,” tutors synthesized informa¬ 
tion from various sources, including CBM-W 
samples and observation notes. They made 
systematic decisions using a data-based 
decision- rubric (available from the first author), 
which includes a series of questions that guide 
tutors to consider possible reasons for insuffi¬ 
cient progress and select appropriate changes. 
They could change the setting and format (e.g., 
by rearranging the learning environment or 
redistributing instructional time across activi¬ 
ties), delivery (e.g., by making intervention 
more explicit), or content (e.g., by changing 
target letters, sounds, or words). More informa¬ 
tion about the frequency and types of changes 
that tutors made are in the Procedures section. 
Step 7 was “make instructional changes based 
on hypotheses.” Tutors recorded changes on a 


form titled “Changes in Instructional Plan” 
(CIP). Step 8 was “repeat Steps 4 through 7.” 

Treatment Condition 

Treatment students received intervention 
in small groups of three to four students. The 
intervention was delivered by trained tutors, 30 
min per session, three times per week, for 12 
weeks, during students’ regular writing instruc¬ 
tion schedules (thus supplanting their usual writ¬ 
ing instruction). 

Control Condition 

Students in the control condition received 
business-as-usual writing instruction. The 
first author and three university researchers 
conducted direct classroom observations of 
writing instruction delivered by the seven 
teachers to control students. These observa¬ 
tions captured characteristics of writing 
instruction identified by Coker and Ritchey 
(2010), including grouping, writing focus, 
writing levels (e.g., word, sentence, or 
passage levels), and type of teacher and 
student responses. Control students received 
varying types of writing instruction depend¬ 
ing on their educational service settings. 
Students with disabilities (n— 14) typically 
received writing instruction individually or in 
small groups by special education teachers, 
focusing on transcription skills at the word 
and sentence levels. Students identified as at 
risk (« = 10) typically received whole-class 
instruction by general education teachers, 
focusing on text generation skills at the 
passage level, in a “writer’s workshop” 
format. According to observers, writing 
instruction in both special education and 
general education settings was less explicit 
and less targeted to specific writing compo¬ 
nents than was treatment writing instruction. 

The first author also engaged in e-mail 
exchanges and met with teachers to obtain 
more information about writing assessment 
and instructional decision-making practices. 
General education teachers reported that they 
rarely monitored student writing progress. 
Rather, they kept students’ writing samples 
and sent them home at the end of the term. 
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Special education teachers reported monitoring 
progress using CBM-W story prompts once 
every 2 to 4 weeks depending on writing objec¬ 
tives in the student’s individualized education 
program; however, they did not use CBM-W 
data for ongoing instructional decision making. 
At the end of the study, the first author 
contacted all teachers to offer to share study 
findings and provide intervention training. 

Procedures 

Five doctoral students majoring in educational 
psychology or curriculum and instruction 
collected screening and pre- and posttest data. 
Most examiners were blinded to condition 
except for a few instances when scheduling 
issues prevented blinded administration. 
Before administration began, the first author 
provided a 1-hr training and checked fidelity 
of CBM-W and WJ III administration using 
the AIRS and checklists of essential adminis¬ 
tration components. Fidelity was 100% for 
both measures. Because treatment students 
were exposed to CBM-W frequently (twice 
per week) during the 12 weeks, control 
students also were administered CBM-W 
twice to address the possibility of practice 
effects of progress monitoring. 

Scoring. The examiners scored CBM-W after 
participating in a 1-hr training. First, they 
scored a common CBM-W set together, 
compared their scores, and discussed and 
resolved discrepancies. After each scorer 
reached the reliability criterion of 80% with 
the first author, who had met the reliability 
criterion of 90% with a special education 
professor who had extensive scoring experi¬ 
ence, a blinded package of writing samples 
was distributed to each scorer. The first author 
compared 20% of each scorer’s writing samples 
with her own and calculated interrater agree¬ 
ment (number of agreements divided by 
agreements plus disagreements and multiplied 
by 100). Interrater agreement ranged from 
91.43% to 100% for all scoring procedures. 

The WJ III Spelling subtest was scored by 
the first author, and then 20% of the scored 
subtests was checked by another scorer for 


interscorer reliability. To score Writing 
Fluency and Writing Samples subtests, the 
first author provided a 1-hr training to a 
doctoral student in special education. The 
interrater agreement procedure was conducted 
in a similar way to CBM-W scoring. Mean 
interrater agreement was 100% for Spelling 
and Writing Fluency and 90.29% for Writing 
Samples (range= 85.74% to 92.86%). 

Tutor training. Six undergraduate and graduate 
students studying educational psychology, 
curriculum and instruction, or psychology 
served as tutors and participated in three 
training workshops. At the first workshop, 
tutors learned the overall structure of the early 
writing intervention activities and materials. 
The second workshop covered CBM-W 
administration, scoring, and graphing. The 
third workshop provided a study overview 
and explained DBI steps, including applying 
data-based decision rules. 

Fidelity of implementation. Fidelity of imple¬ 
mentation was assessed for both writing 
intervention and DBI twice: 4 weeks after 
DBI began and then 4 weeks later, when tutors 
made their first instructional decisions. Fidel¬ 
ity of writing intervention was checked twice 
using the writing intervention checklist for all 
seven writing activities for each tutor. Every 
tutor audio-recorded intervention sessions 
comprising all writing activities. Tutors were 
notified that if fidelity was below 80%, they 
would be asked to audio-record their sessions 
again after receiving corrective feedback 
from the first author. At both fidelity checks, 
however, fidelity was high, so additional 
checks were not necessary. Average writing 
intervention fidelity was 93.62 % at the first 
fidelity check (range= 88.89% to 100%) and 
98.66% at the second check (range= 83.33% 
to 95.24%). A total of 36 out of 216 interven¬ 
tion sessions (17%) were checked. 

DBI fidelity was checked twice using the 
AIRS in the areas of CBM administering, 
scoring, and graphing. The first author collected 
additional materials, including audio-recorded 
CBM-W administration files, CBM-W graphs, 
scored CBM-W prompts, and CIPs from tutors. 
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The first author listened to the audio files 
to check CBM-W administration. CBM-W 
scoring was checked by comparing the writing 
prompts scored by each tutor to the same 
prompts scored by the first author. CBM-W 
graphs and CIPs were examined to determine 
whether tutors made timely and appropriate 
instructional decisions. To be “timely and 
appropriate,” tutors had to document instruc¬ 
tional decisions and rationales on the CIP after 
eight data points, and the timing of the decision 
had to match the graphed data. An instructional 
change was deemed appropriate if the tutor pro¬ 
vided a reasonable rationale based on CBM-W 
data, writing samples, and observations. For 
example, if the trend line was less steep than the 
goal line, indicating the need for instructional 
change, tutors could determine the students’ 
needs based on CBM-W samples (e.g., if students 
consistently misspelled “long a” words, content 
changes would be an appropriate change) or 
could follow guidelines provided for making 
changes to setting, format, or delivery on the 
data-based decision making rubric. Average 
DBI fidelity was 95.68% at the first check 
(range= 90.90% to 100%) and 95.67% at the 
second check (range= 82.86% to 100%). 

Two raters discussed rating criteria for 
each item on the writing intervention check¬ 
list and the AIRS, listened to an audio- 
recorded session together, compared ratings, 
and discussed discrepancies. After reaching 
the criterion of 80% interrater agreement, 
the raters checked fidelity independently. 
Percentage interrater agreement (agreements 
divided by agreements plus disagreements 
and multiplied by 100) was 98.56% for 
writing intervention and 99.33% for DBI. 

Documenting instructional decisions. Tutors 
documented all instructional decisions based 
on CBM-W data on CIP forms. They made 47 
instructional decisions in total across students, 
averaging about two decisions per student 
(range = 0 to 4). They made 19 goal increases 
(40.43%) and five decisions to keep the 
intervention as is (10.64%). They made 23 
instructional changes (48.94%), including 11 
changes to setting and format by rearranging 
the order or length of writing activities or by 


adding a motivational plan (47.83%), eight 
delivery changes by explaining the activity 
and its purpose more explicitly to the student 
or by providing immediate corrective feed¬ 
back (34.78%), and four content changes by 
selecting target letters and words that were 
better matched to students’ needs (17.39%). 

Ongoing support. All tutors participated in 
weekly 1-hr group meetings, in which they 
discussed issues that arose during the week. 
Each tutor presented his or her student’s 
progress monitoring graph and discussed 
possible factors affecting the student’s 
performance and what to change. The first 
author also checked in with each tutor weekly 
(for about 30 min) to ensure each tutor was 
keeping track of students’ progress and 
implementing DBI as intended. 

Data Analysis 

A mixed (two between, one within) repeated- 
measures multivariate analysis of variance 
(RM-MANOVA; Tabachnick & Fidell, 2007) 
was conducted using posttest scores of both 
dependent measures (CBM-W and WJ III). 
Because treatment students received inter¬ 
vention in small groups, we calculated intra¬ 
class correlations (ICCs) for each dependent 
variable to identify any cluster effects. No 
statistically significant ICCs were found (all 
ps> .05) with most ICC values less than 
.005 (range = .003 to .17). Thus, we used 
student as the unit of analysis. 

Condition (treatment vs. control) and 
special education status (at risk vs. with 
disabilities) were between-subjects factors for 
both measures. Type of scoring procedures 
(WW, WSC, CWS, and CIWS) and subtests 
(Spelling, Writing Fluency, and Writing 
Samples) represented within-subjects factors 
for CBM-W and WJ III, respectively. This 
type of repeated-measures analysis is some¬ 
times referred to as profile analysis (Maxwell 
& Delaney, 2004). RM-MANOVA includes 
both between- and within-subjects sources of 
variance in the analysis, providing a better 
model of the multivariate structure of the 
data and reducing error variance estimates 
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Table 2. Descriptive Statistics for CBM-W Scores of Student Participants. 



Students at risk 


Students with disabilities 



Treatment 

Control 


T reatment 

Control 



("= 

9) 

("= 

10) 

ES 

(n= 13) 

(n= 14) 

ES 






Pretest/ 



Pretest/ 

Measure 

Pretest 

Posttest 3 

Pretest 

Posttest 3 

posttest b 

Pretest Posttest 3 

Pretest Posttest 3 

posttest b 

CBM-W: WW 





0.47/0.80 



0.54/0.74 

M 

18.30 

28.65 

14.63 

22.40 


16.00 25.99 

11.38 19.74 


SD 

6.47 


8.22 



9.07 

7.34 


Skewness 

1.09 


-0.03 



0.94 

0.67 


Kurtosis 

1.77 


-0.41 



0.42 

-0.77 


CBM-W: WSC 








0.83/0.72 

M 

14.78 

25.59 

12.07 

19.75 

0.34/0.74 

14.13 23.74 

7.33 17.90 


SD 

6.56 


8.28 



8.54 

7.31 


Skewness 

1.47 


0.29 



0.98 

0.86 


Kurtosis 

3.29 


0.04 



1.32 

-0.43 


CBM-W: CWS 





0.1 1/1.31 



0.75/1.28 

M 

10.48 

23.04 

9.60 

13.84 


10.72 22.37 

5.26 13.17 


SD 

8.19 


7.62 



7.62 

6.36 


Skewness 

1.91 


0.51 



0.73 

1.41 


Kurtosis 

4.42 


-0.31 



-0.52 

0.89 


CBM-W: CIWS 





-0.28/1.33 



0.92/1.41 

M 

-I.5S 

9.91 

1.00 

-2.08 


1.38 9.76 

-6.29 -2.23 


SD 

9.59 


7.60 



6.66 

9.46 


Skewness 

1.39 


0.27 



0.1 1 

0.18 


Kurtosis 

3.30 


-0.04 



-0.63 

-0.35 



Note. CBM-W = curriculum-based measures in writing; WW = words written; WSC = words spelled correctly; CWS = correct 
word sequences; CIWS = correct minus incorrect word sequences; ES = effect size (Hedges’ g) for data-based instruction-control. 
a Adjusted posttest means after controlling for pretest scores on each dependent variable. b Hedges’ g based on the difference between 
the adjusted posttest means divided by the pooled standard deviations of pretest scores. 


compared to conducting an individual 
ANOVA for each dependent variable. The 
design allows for examination of differential 
treatment effects across different scoring 
procedures (CBM-W) or subtests (WJ III) that 
compose a measure. Standardized residual 
scores for each CBM-W metric and for each 
WJ III subtest (W scores), which place all of 
the measures on the same scale, were used for 
the analysis. Significant multivariate results 
were followed up with univariate independent 
one-way analyses of variance. To estimate 
practical importance of treatment effects, 
effect sizes were calculated using Hedges’ 
g (Hedges & Olkin, 1985). 

Results 

Tables 2 and 3 present descriptive statistics by 
special education status and condition, includ¬ 
ing means, standard deviations, skewness, 


kurtosis, and effect sizes for each dependent 
measure. Pretest descriptive statistics are based 
on raw scores, whereas posttest descriptive 
statistics are based on estimated means adjusted 
for pretest scores. Hedges’ g was calculated by 
differences between the estimated posttest 
means between conditions divided by the 
pooled standard deviations of pretest scores. 

Attrition 

Two treatment students did not complete the 
entire posttest. One student left the study after 
receiving intervention for 1 week, due to a 
conflict with the school schedule, so we had 
neither CBM-W nor WJ III data for him. The 
other student was not able to complete 
CBM-W because he was absent on the day it 
was administered and was not available for 
makeups. Fewer than 5% of students in the 
sample had missing data, and they were 
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Table 3. Descriptive Statistics for the WJ III W Scores of Student Participants. 




Students at risk 



Students with disabilities 



T reatment 

( n= 9) 

Control 
( n= 10) 

ES 

T reatment 
(n= 13) 

Control 

(n= 14) 

ES 

Measure 

Pretest 

Posttest 3 

Pretest 

Posttest 3 

Pretest/ 

posttest b 

Pretest 

Posttest 3 

Pretest 

Posttest 3 

Pretest/ 

posttest b 

WJ III: Spelling 

Al 

443.22 

453.99 

447.40 

448.10 

-0.23/0.33 

443.20 

445.77 

419.00 

436.29 

1.16/0.45 

SD 

21.04 


13.15 



16.82 


23.45 



Skewness 

-1.41 


0.54 



-1.03 


0.47 



Kurtosis 

2.45 


-0.48 



1.04 


0.61 



WJ III: Writing Fluency 
M 

466.67 

466.52 

447.40 

473.92 

1.78/-0.89 

466.10 

468.97 

461.10 

465.32 

0.91/0.68 

SD 

5.68 


13.15 



6.91 


2.77 



Skewness 

0.48 


0.54 



1.39 


2.92 



Kurtosis 

-0.99 


3.41 



1.00 


9.05 



WJ III: Writing Samples 
M 

451.00 

461.88 

457.60 

465.61 

-0.24/-0.I4 

449.60 

464.14 

428.4 

442.09 

0.68/0.70 

SD 

33.00 


17.64 



25.08 


34.84 



Skewness 

-2.75 


-0.75 



-1.08 


-0.44 



Kurtosis 

7.87 


1.63 



0.88 


-0.54 




Note. WJ III = Woodcock-Johnson III Tests of Achievement; ES = effect size (Hedges’ g) for data-based instruction-control. a Adjusted 
posttest means after controlling for pretest scores on each dependent variable. b Hedges’ g based on the difference between the 
adjusted posttest means divided by the pooled standard deviations of pretest scores. 


missing in a random pattern, indicating that 
proceeding with the analyses was appropriate 
(Tabachnick & Fidell, 2007). 

To examine whether the analyses were 
sensitive to the two missing posttest cases at 
pretest, RM-MANOVAs were conducted with 
the two students included and excluded, with 
treatment condition and special education sta¬ 
tus as the between-subjects factors and type of 
scoring procedures (WW, WSC, CWS, and 
CIWS) or type of subtests (Spelling, Writing 
Fluency, and Writing Samples) as within- 
subjects factors. The RM-MANOVA indicated 
the analyses were not sensitive to the inclusion 
of the missing data for CBM-W but were sensi¬ 
tive for the WJ III, suggesting it would be 
appropriate to exclude the two students from 
the final analysis (statistical results are avail¬ 
able from the first author on request). The two 
students were excluded from the final analyses, 
resulting in a total sample size of 46 students. 

Pretreatment Differences 

RM-MANOVAs showed no statistically sig¬ 
nificant mean differences between conditions 
on students’ writing performance on CBM-W, 
F(l, 42) = 2.855, p= .099, or the WJ III, 


F( T, 42)= 2.17,/)= .148, at pretest. However, 
treatment students showed consistently higher 
means compared to controls, with effect sizes 
(computed using pretest raw scores from 
CBM-W and the WJ III) greater than 0.25 
for all but three of the variables (see Tables 2 
and 3), thus not satisfying baseline equiva¬ 
lence (What Works Clearinghouse, 2014). To 
control for pretreatment differences, a regres¬ 
sion adjustment was applied for all dependent 
variables. First, a regression model was built 
for each measure with the respective pretest 
scores as the independent variable and post¬ 
test scores as the dependent variable. Then, 
we used standardized residuals from each 
model, which removes variance accounting 
for pretest scores from posttest scores, as the 
dependent variables in the RM-MANOVAs. 

Effects of DBI on Writing 
Performance 

We conducted a RM-MANOVA with condition 
(treatment vs. control) and special education 
status (at risk vs. with disabilities) as between- 
subjects factors and type of scoring procedure 
(WW, WSC, CWS, and CIWS) as the within- 
subjects factor to examine treatment effects 
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Table 4. Results of RM-MANOVA on 

CBM-W and the WJ III Performance. 



Effect 

SS 

df 

Wilks’s A 

F 

P 

CBM-W 

Between subjects 

T reatment 

3.857 

1 

— 

5.293 

.026 

SPED 

0.071 

1 

— 

0.097 

.757 

Treatment x SPED 

Error 

2.014 

0.729 

1 

42 

— 

2.764 

.104 

Within subjects 

Scoring type 

— 

3,40 

.999 

0.020 

.996 

Scoring Type x Treatment 

— 

3,40 

.907 

1.370 

.266 

Scoring Type x SPED 

— 

3,40 

.936 

0.909 

.445 

Scoring Type x Treatment x SPED 

— 

3,40 

.933 

0.956 

.423 

WJ III 

Between subjects 

T reatment 

0.483 

1 

— 

1.422 

.240 

SPED 

2.773 

1 

— 

8.162 

.007 

Treatment x SPED 

Error 

2.204 

14.271 

1 

42 

— 

6.487 

.015 

Within subjects 

Subtest type 

— 

2,41 

.999 

0.018 

.982 

Subtest Type x Treatment 

— 

2,41 

.884 

2.696 

.079 

Subtest Type x SPED 

— 

2,41 

.991 

0.179 

.837 

Subtest Type x Treatment x SPED 

— 

2,41 

.907 

2.106 

.135 


Note. RM-MANOVA = repeated-measures analyses of variance; CBM-W = curriculum-based measures in writing; WJ 
III = Woodcock-Johnson III Tests of Achievement; SPED = special education status; SS = sums of squares. 


on students’ writing performance measured by 
CBM-W. Results, presented in Table 4, revealed 
a significant main effect of condition favoring 
the treatment. Because the interaction was not 
statistically significant, a regression equation 
for each CBM-W measure was fitted that 
included the respective pretest measure and 
only the main effects for condition and special 
education status as predictor variables. Posttest 
means were then estimated using the respec¬ 
tive regression models, holding the pretest 
measure constant at the mean. The magnitude 
of the effect size based on estimated posttest 
means was large for all four scoring procedures 
(g= 0.78, 0.74, 1.23, and 1.36, respectively). 
Main effects of special education status and 
type of scoring procedures were not statisti¬ 
cally significant, and there were no significant 
two- or three-way interactions. 

An RM-MANOVA with condition and 
special education status as between-subjects 
factors and type of subtest (Spelling, Writing 


Fluency, and Writing Samples) as the within- 
subjects factor was conducted to examine treat¬ 
ment effects on students’ writing achievement 
as measured by the WJ III. Results, shown in 
Table 4, revealed no reliable main effect of con¬ 
dition. The RM-MANOVA revealed a statisti¬ 
cally significant main effect of special education 
status favoring at-risk students and a significant 
Condition x Special Education Status interaction. 
A regression equation for each WJ 111 subtest 
was fitted that included the respective pretest 
measure, main effects for condition and special 
education status, and the interaction as predic¬ 
tor variables. Posttest means were then esti¬ 
mated using the respective regression models, 
holding the pretest measure constant at the 
mean. The special education status effect size 
based on estimated posttest means was small to 
medium for all three subtests (g= 0.46, 0.50, 
and 0.37, respectively). 

To examine the nature of the interaction, 
follow-up independent t tests were conducted 
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with condition as the independent variable and 
the average of the three standardized residual 
WJ III writing subtest scores as the dependent 
variable for students at risk and with disabili¬ 
ties separately. Significance of the results was 
determined using the Holm-Bonferroni adjust¬ 
ment (Holm, 1979). No statistically significant 
mean difference was found between conditions 
for students at risk, t(17)= -0.78, p— .45. A 
reliable mean difference favoring the treatment 
group was found for students with disabilities, 
t(25)= 3.25, p= .003, with a medium to large 
effect size based on estimated posttest means 
for all three WJ III subtests (see Table 3). 

Discussion 

Students at risk or with disabilities who 
received early writing intervention within a 
DB1 framework showed significantly higher 
writing performance on CBM-W, but not on 
the WJ III, compared to controls. Thus, our 
hypothesis that treatment students would 
outperform control students was partially sup¬ 
ported. These findings corroborate previous 
DBI research in other academic areas— 
specifically, reading, mathematics, and spell¬ 
ing (Capizzi & Fuchs, 2005; D. Fuchs et al., 
1984; L. Fuchs et al., 1989; Stecker & Fuchs, 
2000; Wesson, 1990). 

One or more combinations of factors may 
have contributed to improved CBM-W out¬ 
comes. Tutors (a) implemented research-based 
early writing intervention three times per week 
while (b) monitoring students’ progress using 
CBM-W twice per week and (c) using the DBI 
framework to evaluate instructional effective¬ 
ness and make individualized decisions. All 
three features are supported by previous 
research for enhancing students’ academic 
achievement (Beminger et al., 1997, 1998; 
Graham et al., 2000, 2002; Stecker et al., 
2005). Findings of this study support the use of 
these assessment, intervention, and decision¬ 
making components as a package to improve 
student outcomes in writing. However, given 
apparent pretest differences favoring treatment 
students, these findings should be interpreted 
with caution even though we controlled the 
pretreatment differences statistically. 


Differential Effects of DBI 

Whereas results indicate that students bene¬ 
fited from the treatment regardless of special 
education status on CBM-W, a statistically 
significant difference between conditions on 
the WJ III measures was found only for those 
receiving special education services. Thus, 
findings support the implementation of early 
writing intervention within a DBI framework 
for improving overall writing performance in 
terms of transcription skills (specifically, 
handwriting and spelling) as well as quantity 
and quality of writing for students receiving 
special education services. 

Students with disabilities might have expe¬ 
rienced significant benefits compared to those 
at risk because intervention focusing on tran¬ 
scription might have been better aligned to 
their specific needs. A focus on transcription 
may not have been sufficient for students at 
risk; in fact, tutors expressed the need for 
more text generation activities for students at 
risk. In addition, at-risk control students may 
have experienced benefits from writing 
instruction by general education teachers, 
which focused more on text generation skills, 
leading to higher writing performance than 
among those at risk who received treatment. 

There was no differential effect of treat¬ 
ment depending on CBM-W scoring proce¬ 
dures or the WJ Ill subtests. In addition, no 
significant three-way interaction (Condition x 
Scoring Procedure x Special Education Status) 
was found. Students with intensive early 
writing needs appeared to improve both basic 
and more complex writing skills. 

Limitations and Directions for Future 
Research 

Several features of the current study limit gen- 
eralizable conclusions that can be drawn from 
the findings. First, as mentioned previously, 
although there were no reliable between-group 
pretreatment differences, treatment students 
showed a pattern of higher mean performance 
levels compared to controls. We statistically 
controlled for preexisting group differences, 
but those differences or others related to them 
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but not measured in our analysis could have 
influenced the results. Future researchers 
should consider assigning student participants 
into conditions randomly after matching them 
on important variables, including dependent 
variables or other characteristics (Gersten 
et al., 2005). Second, a relatively small sam¬ 
ple of students from three schools partici¬ 
pated; replication of this research with larger, 
more diverse samples is warranted. 

Students at risk or with disabilities 
who received early writing 
intervention within a DBI 
framework showed significantly 
higher writing performance on 
CBM-W, but not on the WJ III, 
compared to controls. 

Third, it is unclear which dimensions of the 
treatment contributed to enhanced writing per¬ 
formance given that it was delivered as an 
instructional package. Further research is 
needed to identify the “active ingredients”—in 
other words, which of the assessment, interven¬ 
tion, and DBI components are necessary and 
sufficient to lead to improved student outcomes. 
It may be that receiving research-based early 
writing intervention was sufficient to improve 
student outcomes. However, the decision¬ 
making component of DBI, which prompts an 
instructional change when students are not 
making sufficient progress, likely led to even 
greater growth, at least for some students, 
serving as the most crucial component. On aver¬ 
age, tutors made about two instructional changes 
per student, suggesting that the standard inter¬ 
vention was not sufficient for all students. 

Additional research is needed to examine 
the effects of DBI on quality of writing. Four 
CBM-W scoring procedures used in this study 
represent quantitative aspects of writing. The 
WJ III Writing Samples subtest assessed stu¬ 
dents’ writing quality but was limited to sen¬ 
tence-level writing. Because quality is a 
important aspect of written expression, future 
researchers should consider measuring stu¬ 
dents’ writing quality. Given tutors’ comments 
about the need for more text generation activ¬ 


ities, researchers should also consider includ¬ 
ing a broader range of writing activities in 
future early writing intervention studies. In 
addition, this study provides promising evi¬ 
dence of DBI in early writing delivered by 
trained tutors; a logical next step would be to 
investigate the effects of this approach deliv¬ 
ered by in-service teachers in schools. 

Implications for Practice 

Given that individualized instruction is 
an essential feature of special education 
(D. Fuchs et al., 2010), DBI may serve as an 
effective framework to teach students with 
disabilities who have specific needs in 
writing. Teachers should consider explicitly 
teaching transcription skills for struggling 
beginning writers using research-based inter¬ 
ventions. In this study, students received a 
research-based early writing intervention 
that comprised a variety of handwriting and 
spelling activities, which likely contributed to 
students’ improved writing performance. 
However, research-based intervention may 
not be sufficient for all students all of the time. 
In this study, data indicated the need for 
multiple instructional decisions, about 90% of 
which were to either increase a student’s 
goal or change instruction. We strongly 
recommend that teachers collect ongoing 
progress-monitoring data and use those data 
to make instructional decisions based on 
students’ responsiveness to intervention. 

Conclusion 

Previous studies have supported the benefits 
of DBI for improving students’ performance 
in reading, mathematics, and spelling. This 
study extends the DBI literature by providing 
preliminary promising evidence of early 
writing intervention delivered within a DBI 
framework. In particular, this approach 
appeared to be more effective for students 
receiving special education services, at least 
with the types of early writing activities 
included in this study. Further, this study 
shows the importance of research-based early 
writing intervention focusing on handwriting 
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and spelling. Finally, this study adds to the 
literature that supports a systematic approach 
to individualizing intervention, which is 
essential to providing students with disabili¬ 
ties the special education that they need to be 
successful in school and later life. 
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